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Abstract 

We present a method for estimating the complexity of an image 
based on Bennett's concept of logical depth. Bennett identified logical 
depth as the appropriate measure of organized complexity, and hence 
as being better suited to the evaluation of the complexity of objects 
in the physical world. Its use results in a different, and in some sense 
a finer characterization than is obtained through the application of 
the concept of Kolmogorov complexity alone. We use this measure to 
classify images by their information content. The method provides 
a means for classifying and evaluating the complexity of objects 
by way of their visual representations. To the authors' knowledge, 
the method and application inspired by the concept of logical depth 
presented herein are being proposed and implemented for the first time. 

Keywords: information content, Bennett's logical depth, algorithmic com- 
plexity, image classification, algorithmic randomness. 

1 Introduction 



We present a method for assessing and quantifying the information content 
of an image based on the theory of algorithmic information, specifically on 
Bennett's concept of logical depth [lj. It serves as a means for evaluating 
and classifying images by their organized complexity. Images have a number 
of features containing information in the form of pixels. As a representation 
of an object, an image constitutes a description of said object, capturing 
some of its characteristics. 

Algorithmic information theory [131 H] formalizes the concepts of sim- 
plicity and randomness by means of information. Many applications of the 
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theory of algorithmic information have been developed to date, for example 
[SI dU [161 [T71 [22] . For a detailed survey see [151 E]- None, however, seems to 
have exploited the concept of logical depth, which may provide another use- 
ful complexity measure. Logical depth was originally identified with what 
is usually believed to be the right measure for evaluating the complexity of 
real-world objects such as living beings. Hence its alternative designation: 
physical complexity (used by Bennett himself). This is because the concept 
of logical depth takes into account the plausible history of an object. It 
combines the shortest possible description of the object with the time that 
it takes for this description to evolve to its current state. The addition of 
logical depth results in a reasonable characterization of the organizational 
(physical) complexity of an object, which is not to be had by the application 
of 

the concept of algorithmic complexity alone. The main hypothesis of 
this paper is that images can be used to determine the physical complexity 
of an object (or a part of it) [10] at a specific scale and level, or if preferred, 
to determine the complexity of the image containing information about an 
object. And as such, they cannot all be presumed to have the same history, 
but rather to span a wide and measurable spectrum of evolutions leading 
to patterns with different complexities, ranging from the random- looking to 
the highly organized. To test the applicability of the concept of logical depth 
to a real-world problem, we first approximate the shortest description of an 
image by way of currently available lossless compression algorithms. Then 
the decompression times are estimated as an approximation to the logical 
depth of the image. This allows us to generate a relative measure and to 
produce a classification based on this measure. 

The paper is organized as follows: In section[2]the theoretical background 
that will constitute the formal basis for the proposed method is introduced. 
In Section[3]we describe the method and the battery of tests used to evaluate 
it. Finally, in section [5] we present the results followed by the conclusions in 

m 
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2 Theoretical foundation 



2.1 Algorithmic complexity 

The complexity of a string of bits can be defined in terms of algorithmic 



can run the program and make a copy of s (in our case the image of the 
represented object). The information content of a bit string can be defined 
as the length (in bits) of the smallest program which produces the string s. 

In algorithmic information theory a string is algorithmically random if it 
is incompressible. The difference in length between a string and the shortest 
algorithm able to generate it is the string's degree of complexity. A string 
of low complexity is highly compressible, as the information that it contains 
can be encoded in an algorithm much shorter than the string itself, while a 
string of high complexity is hard to compress because, in a fixed language, 
its shortest possible description is itself. 

The algorithmic complexity [I3j HI [HI [18] Kjj(s) of a binary string s 
with respect to a universal Turing machine U is defined as the binary length 
of the shortest program p of length \p\ that produces s as output: 



K however is not a computable function. In other words, there is no 
program which takes a string s as input and produces the integer K(s) as 
output. Since K(s) is the length of the shortest compressed form of s, that 
is, the best possible compression (up to an additive constant), one can ap- 
proximate K by compression means, using currently available lossless com- 
pression programs. The length of the binary compressed version of s is an 
upper bound of its algorithmic complexity and therefore an approximation 
of K(s). Clear introductions to the subject are available in [3| [T5] . 

2.2 Bennett's logical depth 

A measure of the complexity of a string can be arrived at by combining 
the notions of algorithmic information content and time. According to the 
concept of logical depth [U [2] , the complexity of a string is best defined by 
the time that an unfolding process takes to reproduce the string from its 
shortest description. The longer it takes, the more complex. Hence complex 

1 Also known under the names program-size complexity and Kolmogorov-Chaitin com- 
plexity. 




;iven a program producing a string s, a machine 
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objects are those which can be seen as "containing internal evidence of a 
nontrivial causal history." 

Unlike algorithmic complexity, which assigns a high complexity to both 
random and highly organized objects, placing them at the same level, logical 
depth assigns a low complexity to both random and trivial objects. It is thus 
more in keeping with our intuitive sense of the complexity of physical objects, 
because trivial and random objects are intuitively easy to produce, have no 
long history, and unfold quickly. A persuasive case for the convenience of 
the concept of logical depth as a measure of organized complexity, over 
and against algorithmic complexity used by itself, is made in [9j. Bennett's 
main motivation was actually to provide a reasonable means of measuring 
the physical complexity of real-world objects. 

Bennett provides a careful development [l] of the notion of logical depth, 
taking into account near-shortest programs as well as the shortest one-hence 
the significance value-to arrive at a reasonably robust measure. For finite 
strings, one of Bennett's formal approaches to the logical depth of a string 
is defined as follows: 

Let s be a string and d a significance parameter. A string's depth at 
significance d, is given by 

D d (s)=mm{T(p) : (|p| - \p'\ < d) A {U(p) = a)} 

the number of steps T{p) in the computation U(p) = s, with \p'\ the length 
of the shortest program for s, (therefore K(s)). In other words, D^s) is the 
least time T required to compute s from a d-incompressible program p on a 
Turing machine U. 

The simplicity of Bennett's first definition in [I] makes it suitable for an 
initial investigation, serving as a practical approximation to this measure via 
the decompression times of compressed versions using lossless compression 
algorithms. The decompression time of the compressed version of a string 
is a lower bound of Bennett's logical depth. From now on what we have 
identified as the measure approximating the logical depth of a string s will 
be denoted by D(s), with no need of a significance parameter due to the 
fact that we are using Bennett's first, simpler definition. 

Algorithmic complexity and logical depth are intimately related. The lat- 
ter depends on the former because one has to first find the shortest programs 
producing the string and then look for the shortest times (Looking for the 
shortest programs is equivalent to approximating the algorithmic complexity 
of a string.). While the compressed versions of a string are approximations 
of its algorithmic complexity, decompression times are approximations of 
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the logical depth of the string. These approximations depend on the com- 
pression algorithm used. For algorithmic complexity the choice of universal 
Turing machine is bounded by an additive constant (invariance theorem) 
while for logical depth the choices involved are bounded by a multiplicative 
factor [JJ. 

3 Methodology 

An image can be coded as a string s over a finite alphabet, say the bi- 
nary alphabet-a black and white image. We will denote by K c and D c the 
approximations obtained by means of a compression algorithm c. When 
the algorithmic complexity K{s) is approximated by a lossless compression 
algorithm, this approximation corresponds to an upper-bound of K{s) |15j . 

There seems to have been no previous attempt to implement an appli- 
cation of ideas based on Bennett's logical depth to a real-world problem. In 
order to assess the feasibility of an application of the concept to the problem 
of image characterization and classification by complexity, we performed a 
series of experiments of gradually increasing sophistication, starting from 
fully controlled experiments and proceeding to the use of the best known 
compression algorithms over a larger dataset. 

The battery of tests involved a series of images devised to tune and 
verify different aspects of the methodology, and a more realistic dataset, 
indicating whether the results were stable enough to yield the same values 
after each repetition of the experiment and whether they were consistent 
with the theory and consonant with an intuitive sense of a complex vs. a 
simple object. 

The first experiments consisted in controlling all the environmental pa- 
rameters involved, from the data to the compression algorithm, in order to 
test our first attempts to calculate decompression times. A test to measure 
the correlation between image sizes (random vs. uniformly colored images) 
and decompression times was carried out. The results are in section [4.1.11 

A second test in section 14.1.21 consisted of a series of images meant to 
evaluate the change and magnitude of the decompression times when ma- 
nipulating the internal structure of an image. That is, it served to verify 
that the decompression times decreased as expected when the content of a 
random image was artificially manipulated to make it more simple. This 
consisted of a set of images in which uniform structures consisting of large 
single-bit strings of a fixed size were randomly inserted into images originally 
containing only pseudo-randomly generated pixels. 
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The framework entailed the use of a toy compression program involving 
an algorithm grouping runs of the same bit replaced by a couple of values: 
the replaced bit followed by the number of times the bit was found. No 
further allowances were made, either for dealing with special cases or for 
detecting any other kinds of patterns. We wanted the algorithm and the 
data to be as simple as possible, to be fully controlled in every detail so 
we could better understand the relation between an increase in the size of a 
structure and the decompression time. It consisted of the series of computer- 
generated random images shown in Figure 1. 




Figure 1: Inserting increasingly larger (white) regular blocks into an image 
originally containing randomly distributed black and white pixels. 

This was a way of injecting structure into the images in order to study 
the behavior of the compression algorithm and ensure control of the variables 
involved so as to better understand the next set of results. 

A procedure described in 14. 1.41 was devised to test the case complemen- 
tary to the previous test- to verify that the decompression times increased 
when the content of an image was artificially manipulated, transforming it 
from a simple state (an all-white image) to a more complex state by gen- 
erating images with an increasing number of structures. A collection of 
one hundred images with an increasing number of straight lines randomly 
inserted was artificially generated (see Figure 2) . The process led to interest- 
ing results described in 14.1,41 demonstrating the robustness of the method. 

Three more tests to calibrate the measure based on logical depth are 
proposed in I4.2| with three different series of images. The first two series of 
images were computer generated. The third one was a set of pictures of a 
real-world object. The description of the 3 series follows: 

1. A series of images consisting of random points converted to black and 
white with a probability of a point's existing at any given location 
having a certain value depending on a threshold. This can be seen 
as variation in information content, since the ratio of black and white 
dots varies from high to low from image to image depending on the 
threshold value. 
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Figure 2: The number of lines grows as 2n 2 with n the image number from 
to 99. Introducing lines at this rate allows us to start with a uniformly 
white image and end up with the desired, nearly all-black uniformly colored 
image in 100 steps. 

2. Cellular automaton with elementary rule number 30 (in Wolfram's 
enumeration [21]). then another image with the same automaton su- 
perposed upon itself (rotated 90°), followed by the same automaton 
to which was applied a function introducing random bits to 50 per- 
cent of the image pixels. We expect a greater standard deviation for 
the series of cellular automata because they come in pairs (meaning 
each image comes with its inverse), and permutations should be com- 
mon and ought to occur between these pairs because, intuitively, they 
should have the same complexity. This guess will be statistically con- 
firmed in the result^ section. 

3. A wall and the same wall but viewed from a closer vantage point. 

The last test in !4.3l is the main result of the paper, showing the stability of 
the method and the classification result. The experiment was performed on 
a dataset of randomly chosen pictures, 56 black-and-white pictures coming 
from different sources and representing objects of all kind^E Some were pic- 
tures of actual objects produced by nature and by humans, such as car plans, 
pictures of faces, handwriting, drawings, walls, insects, and so on. Others 
were computer-generated, such as straight lines, Peano curves, fractals, cel- 
lular automata, monochrome and pseudo-randomly generated images. 

The selection of images in section 14.31 was made by hand, bearing in 
mind the goal of including a large variety of objects covering a range of 
seemingly different complexities. We chose images of faces, people, engines 

2 The images are available online under the paper title at 
http : //www.mathrix. org/ experimental AIT/ 
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and electronic boards, images singled out for their high degree of complexity, 
each being usually the result of a relatively long history, whether human 
artifacts or long-lived natural entities. The image bearing the name "table" 
is a table of numerical values- a computer spreadsheet- likely characterized 
by a significant level of complexity. Images tagged "people" are pictures of 
a group of people taken from a distance. The tag "inv" following picture 
names indicates that they are the color inversions of other images in the 
same dataset that we expected would be close in complexity, and they are 
presented side- by-side with their non- inverted versions. "Writing" refers to 
handwriting by a human being, which should also have a significant level 
of complexity, a level of complexity approaching that of other handwritten 
pieces such as the image tagged "formulae," which depicts formulae rather 
than words. "Watch" is the internal engine of a watch. "Cpu A" is a 
picture of a microprocessor showing less detail than "cpu B". "Escher" 
is a painting of tiles by Escher. "Paper" is a corrugated sheet of paper. 
"Shadows B" are the shadows produced by sea tides. "Symmetric B" is 
"shadows B" repeated 4 times. "Symmetric A" is "symmetric B" repeated 
4 times. "Rectangle C" is "rectangle B" repeated 4 times. Those images 
tagged "Peano curve" are space filling curves. "Periodic" and "Alternated" 
are of a similar type, and we thought they would have low complexity, being 
simple images. "Random" was a computer-generated image, technically 
pseudo-random. This randomly-generated picture will serve to illustrate the 
main known difference between algorithmic complexity and logical depth, 
i.e., based on its algorithmic (program-size) complexity the random image 
should be ranked at the top, whereas its logical depth would place it at the 
bottom. All pictures were randomly chosen from the web, transformed to 
B&W and reduced to 600 x 600 pixelaB 

3.1 Towards the choice of lossless compression algorithm 

Deflated is a compliant lossless compressor and decompressor available 
within the zlib package. Deflate compression compresses data using a com- 
bination of the Lempel-Ziv coding algorithm [23j and Huffman coding [12] . 
It is one of the most widely- used compression encoding systems. 

Huffman coding assigns short codewords to those input blocks with high 

3 Unlike the printed version, if seen in electronic form the images can be zoomed in, 
allowing better visualization. They are also available online under the paper title at 
http : //www. algorithmicnature . org 

4 RFC1951: Deflate Compressed Data Format Specification version f.3 
http : 77www . w3 ■ or g/Graphics/PMG/RF C- 1951 | 
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probabilities and long codewords to those with low probabilities. In other 
words, the compressor encodes more frequent sequences with a few bytes 
and spends more bytes only on rare sequences. 

The Lempel-Ziv coding algorithm builds a dictionary and encodes the 
string by blocks using symbols in the dictionary. The Lempel-Ziv algorithm 
leads to actual compression when the input data is sufficiently large and 
is characterized by sufficient redundancy (patterns) [20]. The Lempel-Ziv 
compression algorithm provides upper bounds to K. 

3.2 Compression method 

Since digital images are just strings of values arranged in a special way, 
one can use image compression techniques to approximate the algorithmic 
complexity of an image. And through its image, the algorithmic complexity 
of the object represented by the image (at the scale and level of detail of the 
said image). 

A representative sample of lossless image-compression algorithms (GIF, 
TIFF and JPG 2000) was tested in order to compare the decompression run- 
times of the images being studied. Although we experimented with these 
lossless compression algorithms, we ended up choosing PNG for several rea- 
sons, including its stability and the flexibility it affords, given that it permits 
the use of open-source developed optimizers for further compression. 

Portable Network Graphics (PNG) is a bitmapped image format that 
employs lossless data compression. It uses a 2-stage compression process, 
a pre-compression or filtering, and Deflate. The filtering process is partic- 
ularly important in relating separate rows, since Deflate on its own has no 
understanding that an image is a bi-dimensional array, and instead just sees 
the image data as a stream of bytes. 

There are several types of filters embedded in image compression algo- 
rithms which exploit regularities found in a stream of data based on the 
corresponding byte of the pixel to the left, above, above and to the left, or 
any combination thereof, and encode the difference between the predicted 
value and the actual value. The number of combinations tested is finite and 
limited by the compression algorithm. By applying different filters a lossless 
image compression algorithm uses the data contained in the pixels x±, . . . , xs 
to predict the value of another pixel xg (see Figure 3) that may save space, 
enabling the compression of the said image without losing any information. 
Yet one can optimize the search for a successful combination by setting the 
compression algorithm in such a way that it tries harder, and spend more 
time trying to more effectively compress the data. 
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Figure 3: Image pixel neighborhood. 



Compression can be further improved by so-called PNG-optimizers us- 
ing more filter methods and several other lossless data compression algo- 
rithms. Among these optimizers are PngcrustEl and AdvanceCOMpH, two 
of the most popular open-source optimizers. They tried various compres- 
sion methods and were able to reduce the PNG files by about 10 to 20% 
of their original length. AdvanceCOMP recompresses PNG files (and other 
file formats) using the Deflate 7-Zip implementation. The 7-Zip Deflate en- 
coder effectively extends the scope of Deflate further by performing a much 
more detailed search of compression possibilities at the expense of signifi- 
cant further processor time spent searching, which for this experiment was 
not a matter of concern. 7-Zip Deflate also uses the LZMA algorithm, an 
improved and optimized version of the LZ77 [23] compression algorithm. 
The LZMA0 algorithm divides the data into packets, each packet describ- 
ing either a single byte or an LZ77 sequence, with its length and distance 
implicitly or explicitly encoded^. 

As a sort of verification, we ran a popular zip-based commercial compres- 
sor, set to the maximum possible compression, over the already compressed 
and optimized files. The zip-based archiver was unable to further compress 
any of the files (they were actually always a little larger in size). 

3.3 Using decompression times to estimate complexity 

Inflate is the decoding process that takes a Deflate bit stream for decom- 
pression and correctly produces the original full-size data file. To decode an 
LZW-compressed file, one needs to know the dictionary encoding the cor- 
responding strings in the original data to be reconstructed. The decoding 
algorithm for LZ77 works by reading a value from the encoded input and 

5 Syntax example: pngcrush -reduce -brute -e ". compressed. png" /testimages/*.png. 
More info: http://pmt.sourceforge.net/pngcrush/ 

6 Syntax example: e.g. advdef -z -4 *. png ('4' indicating the so-called "insane" compres- 
sion according to the developers). More info: http://advancemame.sourceforge.net/ 

7 More technical details are given in |http : //www . 7-zip . o rg/7z . htmT] 

8 A useful website showing a benchmark of compression algorithms is at url- 
http: / /tukaani.org/lzma/benchmarks.html. 
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outputting the corresponding string from the shorter file description. 

It is this decoding information that Inflate uses when importing a PNG 
image for display, so the lengthier the directions for decoding, the longer 
the time it takes. We are interested in these compression/decompression 
processes, particularly the compression size and the decompression time, as 
an approximation of the algorithmic complexity and the logical depth of an 
image. 

The decompression directions for trivial or random-looking objects are 
simple to follow, with the decompression process taking only a small amount 
of time, simply because the compression algorithm either compresses very 
well and the decompression is just straightforward (a trivial case), or does 
not compress at all (a random case). Longer runtimes, however, are usually 
the result of a process following a set of time-consuming decompression 
instructions, hence a complex process. 

3.4 Timing method 

The execution time was given by the Mathematica function Timing. The 
function Timing evaluates an expression and returns a list of the time taken 
in seconds, together with the result obtained. The function includes only 
CPU time spent in the Mathematica kernel. 

The fact that several processes run concurrently in computing systems 
as part of their normal operation is one of the greatest difficulties faced in 
attempting to measure with accuracy the time that a decompression process 
takes, even when it is isolated from all other computer processes. This 
instability is due to the fact that the internal operation of the computer 
comes in several layers, mostly at the operating system level. In order to 
avoid measurement perturbations as much as possible, several stabilizing 
measures were undertaken: 

• Most computer batch processes and operating system services were dis- 
abled, including services such as wireless, bluetooth and energy saving 
features, such as the hard drive sleeping and display dimming fea- 
tured 

Timed on two different computers for validation: on a MacBook Intel Core 2 Duo 
2GHz, 2048MB DDR2 667Mhz with a solid-state drive (SSD) and on a MacBook Pro 
Intel Core 2 Duo 2.26Ghz, 4096MB DDR3 1067Mhz with a traditional hard disk drive 
(HDD), both running Mac OS X Version 10.6.1 (Snow Leopard). The MacBook Pro was 
always twice as fast on average. 

10 To understand the number of layers of complexity involved in a modern computing 
system see Tanenbaum's book on operating systems |19| . 
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• The microprocessor was warmed up before each experiment by running 
an equivalent process (e.g. a mock experiment run) before running the 
actual one, in order to have the fan and everything else already working 
optimally (like preheating an oven). 

• The cache memory was cleared after each function call using the Math- 
ematica function ClearSystemCache. No history was saved in RAM 
memory. 

• A different order of images was used for each experiment run. The 
result was averaged with at least 30 runs, each compressing the images 
in a different random order. This helped to define a confidence level 
on the basis of the standard deviation of the runs- when normally 
distributed- a confidence level with which we were satisfied and at 
which we arrived in a reasonable amount of time. Further runs showed 
no further improvement. Measurement stabilization was reached after 
about 20 to 30 runs. 

The presence of some perturbations in the time measure values was un- 
avoidable due to lack of complete control over all the computing system 
parameters. As the following battery of tests will show, one can reduce and 
statistically curtail the impact of this uncertainty to reasonable levels. 

4 Results 

4.1 Controlled experiments 

4.1.1 Image size uncertainty variation 



Decompression runtimes Standard deviations 
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Figure 4: Decompression times and standard deviations for monochromatic 
images increasing in size. 
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Figure 5: Decompression times and standard deviations for images with 
(pseudo) random noise increasing in size. 



Figures 4 and 5 show that decompression times using the PNG algorithm 
and the optimizers increase in linear fashion when image sizes increase lin- 
early. The distributions of standard deviations in the same figures show no 
particular tendency apart from suggesting that the standard deviations are 
likely due to random perturbations and not to a bias in the methodology. 

A comparison with Figure 6 shows that standard deviations for images 
containing random noise always remained low, while for uniformly colored 
images standard deviations were significantly higher. 




■ uniform 
random 



xiooViT) 



Figure 6: For uniformly colored images standard deviations of decompres- 
sion times were larger on average. For random images the standard devia- 
tions were smaller and compact. The size of the image seemed to have no 
impact on the behavior of the standard deviations in either case. 



4.1.2 Fully controlled test 

For this test only, we used a toy compression algorithm designed to be as 
simple as possible in order to estimate the uncertainty due to system per- 
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Figure 7: Histogram of the standard deviations for uniformly colored and 
random images increasing in size. 

turbations. We devised a test in which we would have full control of the 
data and the compression algorithm. By understanding the processes we 
expected to be better able to predict compression rates and decompression 
uncertainties, and to quantify the uncertainty of the measurement of decom- 
pression times in the general case (i.e. using other compression algorithms). 

It was found that the algorithmic complexity approximation (the file size) 
decreased linearly at a rate of 1990 bits per image, the same as the number 
of regular inserted bits per image, taking about log 2 (2000) to encode the 
compressed regular string, as would have been theoretically predicted (by 
Shannon's information theory). One can also predict the decompression 
runtime by calculating the slope of the decompression rate. Each insertion 
of 2000 regular bits into the random image took 0.00356 seconds less than 
the preceding one, fitting the estimate computed by the rate ratio. 

The behavior of the standard deviations suggests that the more random 
an image the less stable the decompression time, and the more regular the 
more stable. This may be explained by the number of operations that the 
toy compression algorithm uses for encoding a regular string. In the ex- 
treme case of a uniformly colored image, the code comes up with a single 
loop operation to reproduce the image, taking only one unit of time. On the 
other hand, when an image is random, wholly or partially, the number of 
operations is larger because the algorithm finds small patterns everywhere, 
patterns that have a timing cost per operation performed. Each loop op- 
eration using the Table function in Mathematica takes 0.000025 seconds on 
average, while each iteration takes only about 0.00012 seconds on average, 
suggesting that it is the number of operations that determines the runtime. 

The standard deviation of the mean 0.0000728 is smaller than the average 
of the standard deviations of each point, which is 0.000381. This shows that 
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Figure 8: Using the toy compression algorithm, the larger the white region 
in an image with random noise, the shorter the decompression runtime and 
the lower the standard deviations. 

runtime perturbations were not significant enough. 
4.1.3 Using the PNG compression algorithm 

This time we proceeded to apply the PNG compression algorithm together 
with the optimizers (Pngcrush and AdvancedCOMP) to the same computer- 
generated images with random noise used in the previous section, confident 
that we understood the variables involved in the previous experiment, and 
that everything seemed to be controlled. 

String compression 

34 000 
32 000 
£ 30 000 

| 2a ooo 

2 6 000 
24 OOD 

10 20 30 40 

number of inserted regular (x2000) bits 

Figure 9: Compression rate for uniformly colored images. As for the con- 
trolled experiment with the toy compression algorithm using the PNG com- 
pression algorithm, the compressed length of the images also decreased in 
a predictable way, as one would have expected for such a controlled experi- 
ment. And they did so at a rate of 310 bits per image. 

Figure 9 shows that the PNG0 compression algorithm followed the same 
"with Pngcrush and AdvanceCOMP 
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trend as the toy compression algorithm, namely the more uniformity in a 
random image the greater the compression. PNG however achieved greater 
compression ratios compared to the toy compression algorithm, for obvious 
reasons. 

In Figure 9 one can see that there are some minor bumps and hollows all 
along. Their deviation does not, however, seem significantly different from 
a uniform distribution, as shown in Figure 10, suggesting bias toward no 
particular direction. As was the case with the toy compression algorithm, 
using the PNG algorithm too caused algorithmic complexity to decrease as 
expected-upon the insertion of uniform strings. 

Anomalies histogram 



- 100 --50 50 100 150 200 250 

Figure 10: Anomalies found in the decompression times deviating from the 
main slope seem to behave randomly, suggesting no methodological bias. 



String decompression runtimes Standard deviations 
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Figure 11: Random images produce smaller standard deviations (notice the 
plot scale). But unlike the toy case, decompression times remained statisti- 
cally the same despite an increase in image size. 

4.1.4 Increasing complexity test 

The first thing worth noticing in Figure 11 (left) is that the standard devia- 
tions remained the same on average, suggesting that they were not related to 
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Image decompress ion, runtimes 
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Figure 12: Compressed path: Going from all-white to all-black by randomly 
inserting black lines. 

the image, but rather to external processes. The process of randomly depict- 
ing lines seems to have a low maximum limit of complexity, which is rapidly 
reached. Nevertheless, eight bins of images with significantly different de- 
compression time values, that is, with non-overlapping standard deviations, 
were identified. What Figure 12 suggests is that increasing the number of 
lines can only lead to a limited maximum degree of complexity bounded by 
a convex curve. The eight significantly different images have been selected 
according to jumps that were maximizing their differences: n = AD /a with 
A = max{.D(Ij)[? € {1, . . . , n}} — mm{D(Ii)\i £ {1, . . . , n}}, with Ii each 
image in the set, and D the logical depth. 

As regards the discontinuity in the graph, one hypothesis is that the 
compression algorithm reaches a threshold favoring some regularities over 
others, producing jumps in the decompression times. For a certain quantity 
of randomly depicted lines, the limit remains stable for a while once reached, 
until the moment when the increasing number of inserted lines fills up the 
space and the configuration reaches its lowest complexity by decompression 
time, at which point it begins to approach a phase in which it resembles a 
uniformly colored image. 

The decompression time depicted in Figure 12 turned out to be very 
interesting, suggesting what one might expect for this kind of experiment: 
a path traced between two monochromatic (fully colored) images, an ini- 
tial all-white image succeeded by images of increasing complexity compris- 
ing random lines, and ending at the horizontal departure line in a final 
monochromatic, almost all-black image, when the space becomes entirely 
filled with black lines. 
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4.2 Calibration tests 



The following are the classifications of series 1, 2 and 3 according to their 
decompression runtimes as described in the methodology section [3l 
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Figure 13: Series 1 classification: Random points with different densities. 





Figure 15: Series 3 classification: Zooming in a wall. 



• Series 1 classification (Figure 13): Images labeled 1 and 2 never came 
last, while images 4 and 5 never came first. The first half of the images 
remained the same run after run, while the second half also remained 
stable, with some occasional permutations among the intermediate 
elements of each half, but never with the extremes exchanging places. 

• Series 2 classification (Figure 14): A cellular automaton superposed 
upon itself rotated 90°, with random bits inserted. The runs not only 
mostly never changed, the procedure consistently sorted the images 
into pairs, grouping images with similar characteristics (to which the 
same function was applied). The classification seems consonant with 
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series number 


description 


Std. deviation 


2 
3 
1 


cellular automata 
wall zooming 
random points 


0.000556 
0.00046 
0.000349 



Table 1: Standard deviations per series sorted from greatest to lowest stan- 
dard deviation. 

what one may take to be the more random vs. the less random. All 
of them had low physical complexity in general, given their values, 
which accords with what is believed about cellular automaton rule 
30, i.e. that it is known to behave randomly [21 j . with some regular 
structures on the left. 
• Series 3 classification (Figure 15): A wall and the same wall but viewed 
from a closer vantage point. It turned out to be highly stable after 
several runs, sorting the series of images from the farthest to the clos- 
est picture of the wall, suggesting that pictures taken from a greater 
distance captured more of the structure of the wall, while zooming in 
without also increasing the resolution meant losing detail and struc- 
ture. The closest picture always came last, while the farthest always 
came first. There were only occasional permutations among those in 
between. 

As was expected (see section [3]), a greater standard deviation for the 
series of cellular automata was found due to the image pairing. Each image 
occurred in tandem with its inverse, but permutations were common-and 
expected-between members of pairs, which intuition tells us ought to have 
the same complexity. The expected permutations produced a larger stan- 
dard deviation. 

4.3 Compression length ranking 

The following experiments were carried out as described in section [3] using 56 
black-and-white images spanning a range of objects, each seemingly having 
a different complexity which we could intuitively gauge more or less accu- 
ratehf^l. Each image has a very short description, followed by the image 

12 The images are available online under the paper title at 
http : //www. algorithmicnature . org 
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itself and by the approximate 
for our general compression al; 



values of K c (Figure 16) and D c (Figure 17) 
gorithm c. 




Figure 16: Ranking by compressed lengths: K c {i n ) < K c {i n+ \) with K c (i) 
the approximation to the algorithmic complexity K{i) of the image i by 
means of the PNG compression algorithm aided by Pngcrush and Advance- 
COMP. These classifications are available online for better visualization at 



http://www.algorithmicnature.org under the paper title. 



The classification in Figure 16 presents the images ranked according to 
their compressed lengths using PNG image compression together with the 
PNG optimizers. It goes from larger to smaller, and as can be seen, the 
more random-looking or highly structured, the better classified, while triv- 
ial images come last. One can verify that the procedure is invariant to simple 
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transformations, such as inversions and complementations, since images and 
their symmetric versions are always next to (or close to) each other, indi- 
cating that the compression algorithm behaves as one would expect (that 
is, that inverting colors, for example, has no impact on the resultant com- 
pressed length). One would hardly say, however, that a random image is 
physically complex (random processes, like a gas filling a room, unfold in al- 
most zero time and seem to require no great computational power), which is 
why plain algorithmic complexity does not help to distinguish between com- 
plexity associated with randomness and complexity associated with highly 
structured objects. 

4.4 Decompression times ranking 

The classification in Figure 17 goes from greater to smaller logical depth 
based on the decompression runtimes. The number at the bottom is the 
time in seconds that the PNG algorithm took to decompress the images 
when applied to their compressed versions. Images we gauged to have the 
highest physical complexity came first. It is also worth mentioning that the 
images and their inverted versions remained close to each other, meaning 
that they were always in the same complexity group, which too is just what 
we expected. This also indicates the soundness of the procedure, since im- 
ages and their inversions should be equal in complexity, and therefore equal 
in complexity to the object as well, at a commensurate scale and level of 
detail. 

Figure 18 shows some of the jumps seen before in the experiments in 
section I4.1.4i We think they may be due to the behavior of the compression 
algorithm. The compression algorithm applies several filters, and it may 
favor some regularities over others that are better (faster) decoded after a 
certain threshold is reached, producing these jumps. 

Images were grouped in 8 significantly different groups (with different 
decompression times and therefore seemingly different logical depths). For- 
mally, x{g{) ± <x(<7i) > x(gj) ± a(gj) for any two different group indexes 
i,j € {A,B,...,H}. 

The average difference between the largest and the smallest K c value 
was about 62090 bits, while the average difference between the largest and 
the smallest D c was 0.095 seconds. The largest calculated compressed image 
size (image 1) was 159.8 times larger than the shortest compressed image 
size (image 56). The largest evaluated decompression time (image 5) was 
2.46 times greater than the shortest calculated decompression time (image 
53). 
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Figure 17: Ranking by decompression times: D c (i n ) < D c (i n+ i) with D c (i) 
the approximation to the logical depth D{i) of the image with number i 
according to the indexing from the previous classification for K c , by means 
of the PNG compression algorithm aided by Pngcrush and AdvanceCOMP. 



No significant statistical correlation between K c and D c was found, indi- 
cating that K c and D c are actually two different measures. Figures 19 and 
20 illustrate the classification values from K c to 8 significantly different de- 
compression time (D c ) groups according to each of the two measures. The 
ranking of images based on their decompression times corresponds to the 
intuitive ranking resulting from a visual inspection, with things like micro- 
processors, human faces, cities, engines and fractals figuring at the top as 
the most complex objects, as shown in Figure 21 (group A), while random- 
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Image decompression runtimes 
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Figure 18: Mean and standard deviations of the decompression times of the 
56 images. 




Figure 19: Rank comparison between K c and D c as functions of the chosen 
images having significantly different D c values. 



looking images, ranked high by algorithmic complexity, were ranked low 
(group G) according to the logical depth expectation, being classified next 
to trivial images such as the uniformly colored (group H), and thus indicat- 
ing the characteristic feature of the measure of logical depth. A gradation 
of different complexities can be found in the groups between, gradually in- 
creasing in complexity from bottom to top. 
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Figure 20: K to D mapping by groups indicating the complexity group (by 
logical depth) to which each image instance on the right belongs. 



5 Conclusions and further work 

Extensive experiments were conducted. Along the way we think we have 
shown that: 

1. Ideas in the spirit of Bennett's logical depth can be implemented to 
approach a real-world characterization and classification problem, in- 
stead of simply using the concept of algorithmic complexity alone, 
which distinguishes only between random and trivial objects but does 
not separate random from structured objects. 
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2. Based on the study of several cases and the testing of several compres- 
sion algorithms, the method described in this paper has been shown 
to work, and to be of use in identifying and classifying images by their 
apparent physical complexity. 

3. The procedure described herein constitutes an unsupervised method 
for evaluating the information content of an image by physical com- 
plexity. 

4. The method involving the use of decompression times yields a reason- 
able measure of complexity that is different from the measure obtained 
by considering algorithmic complexity alone, while being in accordance 
with intuitive expectations of greater and lesser complexity. 
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It remains to investigate how this procedure may be improved upon 
to accurately follow Bennett's stricter definitions of Logical Depth. For 
example, it could be that by allowing a couple of additional bits in the 
compressed versions of the images one gets shorter decompression times, 
and we think experiments in this connection are possible. To stay true to 
the spirit of Bennett, one may use several compressors c±, . . . ,c n on strings 
Si,...,s m and compute D c .^ s .^ for all 1 < i < n and 1 < j < m. 

Then, an object Sh is assumed to be more complex than an object Sk if 
D Ci ( X j) > D c% (x k ) for all 1 < i < n and a partial order is thus obtained, from 
which a total order could be obtained by calculating the (e.g. harmonic) 
average of the decompression times. 
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